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Title of the Invention 

Single-Chip Architecture For Shared-Memory Router 

This application claims priority of Provisional Application No. 60/060,628, 
hereby incorporated by reference as if fully set forth herein. 

Background of the Invention 

1. Field of the Invention 



router. 



This invention relates to a single-chip architecture for a shared-memory 
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1 2. Related Art 
2 

3 In a packet-switched network, a "router" is a device which receives pack- 

4 ets on one or more input interfaces and which outputs those packets on one of a plural- 

5 ity of output interfaces, so as to move those packets within the network from a source 

6 device to a destination device. Each packet includes header information which indi- 

7 cates the destination device (and other information), and the router includes routing in- 

8 formation which associates an output interface with information about the destination 

9 device (possibly with other information). The router can also perform other operations 

10 on packets, such as rewriting the packets 5 headers according to their routing protocol or 

1 1 to reencapsulate the packets from a first routing protocol to a second routing protocol. 

12 

13 It is advantageous for routers to operate as quickly as possible, so that as 

14 many packets as possible can be switched in a unit time. Because routers are nearly 

1 5 ubiquitous in packet-switched networks, it is also advantageous for routers to occupy as 

1 6 little space as possible and to be easily integrated into a networking system. For exam- 

17 pie, implementing a router on a single chip (that is, a single integrated circuit) would be 

18 particularly advantageous. 

19 

20 In this regard, one problem which has arisen in the art is that individual in- 

21 tegrated circuits and their packages are relatively limited in resources needed to imple- 

22 ment a router. In particular, individual chips have only a relatively limited number of 

23 pins, a relatively limited die area, and a relatively limited amount of power available for 

24 operation. These limitations severely limit the possibility of providing a useful router on 

25 a single chip. Routing devices generally need relatively more input and output ports 
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1 (thus requiring relatively more pins), relatively more lookup table space (thus requiring 

2 relatively larger die size for memory), relatively more packet buffering space (thus re- 

3 quiring relatively larger die size for memory), and relatively more packets routed in unit 

4 time (thus requiring relatively larger die size for processing ability and relatively larger 

5 power dissipation for speed). 

6 

7 Accordingly, it would be advantageous to provide a single-chip router. 

8 This advantage is achieved in an embodiment of the invention in which a router inte- 

9 grated on a single chip shares memory among packet buffers for receiving packets, 

I o packet buffers for transmitting packets, and packet header buffers for packet forwarding 

I I lookup, and in which accesses to that shared memory are multiplexed and prioritized to 
1 2 maximize throughput and minimize routing latency. 

13 

14 Summary of the Invention 

15 

1 6 The invention provides a single-chip router. The router includes a memory 

17 shared among packet buffers for receiving packets, packet buffers for transmitting pack- 
is ets, and packet header buffers for packet forwarding lookup. Accesses to that shared 

1 9 memory are multiplexed and prioritized. Packet reception is performed with relatively 

20 high priority, packet transmission is performed with medium priority, and packet for- 

21 warding lookup is performed with relatively low priority. 

22 

23 In a preferred embodiment, the single-chip router includes circuits for seri- 

24 ally receiving packet header information, converting that information into a parallel for- 

25 mat for transmission to an SRAM for lookup, and queuing input packets for later for- 
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warding at an output port. Similarly, in a preferred embodiment, the single-chip router 
includes circuits for queuing output packets for transmission at an output port, receiving 
packet forwarding information from the SRAM in a parallel format, and converting 
packet header information from output packets into a serial format for transmission. The 
single-chip router also includes a region in its shared memory for a packet forwarding 
table, and circuits for performing forwarding lookup responsive to packet header infor- 
mation. 

Brief Description of the Drawings 

Figure 1 shows a block diagram of a system including a single-chip router. 

Figure 2 shows a process flow diagram of a method for operating a system 
including a single-chip router. 

Detailed Description of the Preferred Embodiment 

In the following description, a preferred embodiment of the invention is 
described with regard to preferred process steps and data structures. Those skilled in the 
art would recognize after perusal of this application that embodiments of the invention 
can be implemented using circuits adapted to particular process steps and data structures 
described herein, and that implementation of the process steps and data structures de- 
scribed herein would not require undue experimentation or further invention. 
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1 System Elements 

2 

. 3 Figure 1 shows a block diagram of a system including a single-chip router. 

4 

5 A system 100 includes a single-chip router 110, a memory 120 coupled to 

6 the router 110 using a memory bus 121, and a processor 130 coupled to the router 110 

7 using a processor bus 131. 

8 

9 The router 110 includes a plurality of input ports 111, a plurality of output 

10 ports 112, a memory port 113 coupled to the memory bus 121, a processor port 114 cou- 

11 pled to the processor bus 131, and a set of internal memory and internal processing cir- 
'12 ~ " cuits integrated into a single monolithic integrated circuit on at least one side of a silicon 
13 die. 

14 

15 In a preferred embodiment, the memory 120 includes an SRAM, and the 

1 6 memory bus 130 includes a 256 bit wide bus operating at about 125 megahertz, so as to 

1 7 provide 32 gigabits per second full duplex communication (that is, both to and from the 

1 8 memory 120). The memory 120 includes sufficient storage to record a set of packets 140 

1 9 which are received from the input ports 1 1 1 and which are in transit to the output ports 

20 112. 
21 

22 The memory port 1 13 includes a memory data register 151 having 64 eight- 

23 bit bytes disposed in a set of four groups of 16 eight-bit bytes, and disposed for receiv- 

24 ing data from selected registers of the memory 120 (such as in memory read operations) 

25 and for transmitting data to selected registers of the memory 120 (such as in memory 
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write operations). The memory port 113 also includes a memory address register 152 for 
selecting the registers of the memory 120 to be read or written. 

Packet Receive Circuits 

Each one of the input ports 1 1 1 is coupled to an input MAC circuit 161, for 
receiving a set of packets 140 from the input port 111, recognizing a MAC address of the 
sending device, recognizing a MAC address of the router 110 (as the receiving device), 
and coupling the packets 140 to an input packet queue 162. In a preferred embodiment, 
the input MAC circuit 161 receives the packets 140 in a bit serial format and outputs 
them to the input packet queue 162 as a sequence of eight-bit bytes. 

The input packet queue 162 includes a shift register, for receiving the se- 
quence of eight-bit bytes in serial from the input MAC circuit 161, and for transmitting a 
set of 256 bits (that is, 64 eight-bit bytes) in parallel to the memory data port 151. In a 
preferred embodiment, the input packet queue 162 is double-buffered; that is, it includes 
two separate shift registers, one of which can be reading packets 140 in serial from the 
input MAC circuit 161 while the other can be writing packets 140 in parallel to the 
memory data port 151. 

The input packet queue 162 is coupled to a receive request circuit 163, for 
determining that the packet 140 has been received (or partially received, if more than 
256 bits in length), and for signalling the memory 120 to read the packet 140 from the ' 
input packet queue 162. The receive request circuit 163 is coupled to the memory ad- 
dress register 152 and to a control signal for the memory 120. 



WO 99/17182 



7 



PCT/US98/20627 



Packet Transmit Circuits 

Similar to the input ports 111, each one of the output ports 112 is coupled 
to an output MAC circuit 171, for transmitting a set of packets 140 from the output port 
112, adding a MAC address for the router 110 (as the sending device), adding a MAC 
address for the receiving device, and coupling the packets 140 from an output packet 
queue 172. In a preferred embodiment, the output MAC circuit 161 receives the packets 
140 as, a sequence of eight-bit bytes and outputs them from the output packet queue 
172 in a bit serial format. 

- Sirnilarjo the input packet queue 162, the output packet queue 172 in- 
cludes a shift register, for receiving a set of 256 bits (that is, 64 eight-bit bytes) in parallel 
from the memory data port 151, and for transmitting a sequence of eight-bit bytes in se- 
rial to the output MAC circuit 171. In a preferred embodiment, the output packet queue 
172 is double-buffered; that is, it includes two separate shift registers, one of which can 
be reading packets 140 in parallel from the memory data port 151 while the other can be 
writing packets 140 in serial to the output MAC circuit 171. 

Similar to the input request circuit 163, the output packet queue 172 is 
coupled to a transmit request circuit 173, for determining that the packet 140- is ready to 
be transmitted (or partially ready, if more than 256 bits in length), and for signaling the 
memory 120 to write the packet 140 to the output packet queue 162. The transmit re- 
quest circuit 173 is coupled to the memory address register 152 and to a control signal 
for the memory 120. 
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1 

2 Packet Address Lookup Circuits 
3 

4 The input packet queue 162 is also coupled to a packet header queue 182, 

5 for isolating a packet header 141 for the packet 140 and for performing address lookup 

6 for that packet header 141. In a preferred embodiment, the packet header queue 182 re- 

7 ceives the packet header 141 in parallel from the input packet queue 162. 

8 

9 The packet header queue 182 includes a shift register, for receiving a set of 

10 256 bits (that is, 64 eight-bit bytes) in parallel from the input packet queue 162, and for 

11 coupling the packet header 141 to an address request circuit 183. In a preferred em- 
-12 - -bodiment, the packet header queue 182 is double-buffered; that is, it includes two sepa- 

13 rate shift registers, one of which can be reading packet headers 141 in parallel from the 

14 input packet queue 162 while the other can be coupling packet headers 141 to the ad- 

1 5 dress request circuit 183. 

16 

17 The address request circuit 183 includes a hash circuit 184 for determining 

18 a hash address for packet lookup in the memory 120. The hash circuit 184 is coupled to 

1 9 the memory address register 152 for supplying a hash address to the memory 120 for per- 

20 forming packet lookup, The address request circuit 183 is also coupled to a control sig- 

21 nal for the memory 120. 

22 

23 In a preferred embodiment, the hash circuit 184 is- responsive to a (source, 

24 destination) pair in the packet header 141, such as described in detail in the following 

25 co-pending patent applications: 
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o U.S. Application Serial No. 08/581,134, titled "Method For Traffic Management, 
Traffic Prioritization, Access Control, and Packet Forwarding in a Datagram Com- 
puter Network", filed December 29, 1995, in the name of inventors David R. 
Cheriton and Andreas V. Bechtolsheim, assigned to Cisco Technology, Inc., at- 
torney docket number CIS-019; 

o U.S. Application Serial No. 08/655,429, titled "Network Flow Switching and 
Flow Data Export", filed May 28, 1996, in the name of inventors Darren Kerr and 
Barry Bruins, and assigned to Cisco Technology, Inc., attorney docket number 
CIS-016;and 

o U.S. Application Serial No. 08/771,438, titled "Network Flow Switching and" 
Flow Data Export", filed December 20, 1996, in the name of inventors Darren 
Kerr and Barry Bruins, assigned to Cisco Technology, Inc., attorney docket num- 
ber CIS-017. 

These patent applications are collectively referred to herein as the 
"Netflow Switching Disclosures". Each of these applications is hereby incorporated by 
reference as if fully set forth herein. 

The memory 120 responds to the hash address by delivering a set of 
packet lookup information to the memory data register 151, which is coupled to the' 
packet header queue 182. The address request circuit 183 also includes a comparator 
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1 185 for determining which of several packet lookup responses coupled to the packet 

2 header queue 182 is associated with the actual packet header 141. 
3 

4 The packet header queue 182 is also coupled to the processor bus 131, for 

5 coupling packet headers 141 and packet lookup information to the processor 130 for 

6 extraordinary processing. Thus, when the router 110 is unable to process the packet 

7 140, or processing the packet 140 requires more flexibility than available to the router 

8 110 and the memory 120, the packet header 141 is coupled to the processor 130 for ex- 

9 traordinary processing. 

10 

11 In a preferred embodiment, such extraordinary processing can include en- 

1.2 hanced packet forwarding and traffic management services such as access control, mul- 

13 ticast packet processing, random early discard, and other known packet processing 

14 services. 

15 

16 System Operation 

17 

1 8 Figure 2 shows a process flow diagram of a method for operating a system 

1 9 including a single-chip router. 

20 

21 A method 200 includes a set of flow points to be noted, and steps to be 

22 executed, cooperatively by the system 100, including the router 110, the memory 120, 

23 and the processor 130. 

24 
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1 At a flow point 210, an incoming packet 140 is received at one of the input 

2 ports 111. 
3 

4 At a step 221, the input MAC circuit 161 receives the packet 140 and both 

5 recognizes the MAC address for the sending device, and confirms that the MAC address 

6 for the receiving device is the router 110. 

7 

8 At a step 222, the input packet queue 162 receives the packet 140. 

9 

10 At a step 223, the receive request circuit 163 determines a location in the 

1 1 memory 120 for the packet 140, and signals the memory 120 to receive the packet 140. 

- - - - 12 

13 At a step 224, the packet 140 is read into the shared memory 120 from the 

14 input packet queue 162. 

15 

16 At a flow point 230, the packet 140 is ready to be routed. 

17 

18 At a step 241, the packet header 141 for the packet 140 is coupled from the 

19 input packet queue 162 to the packet header queue 182. 

20 

21 At a step 242i the hash circuit 184 determines a hash address for the 

22 (source, destination) pair in the packet header 141, as described in the Netflow Switch- 

23 ing Disclosures, hereby incorporated by reference. 



24 
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At a step 243, the address request circuit 163 couples the packet header 
151 to the memory data register 151, couples the hash address to the memory address 
register 152, and signals the memory 120 to perform a packet address lookup. 

At a step 244, the memory 120 performs the packet address lookup and 
returns its packet lookup results to the memory data register 151. In a preferred em- 
bodiment, the memory 120 is disposed as a four-way set-associative memory responsive 
to the hash address provided by the hash circuit 184, so there are four packet lookup 
results. 

At a step 245, the comparator 185 determines which one of the four packet 
lookup results is valid for the (source, destination) pair in the packet header 141, and se- 
lects that one of the four packet lookup results for packet forwarding. 

At a flow point 250, the packet 140 is ready to be transmitted in response 
to the packet lookup results. 

At a step 261, the transmit request circuit 173 determines the location in 
the memory 120 for the packet 140, and signals the memory 120 to transmit the packet 
140. 

At a step 262, the packet 140 is read from the shared memory 120 into the 
transmit packet queue 172. 
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1 At a step 263, the output MAC circuit 171 both recognizes the MAC ad- 

2 dress for the sending device, and adds the MAC address for the receiving device (the 

3 router 110 itself), and transmits the outgoing packet 140 on the output port 112. 



4 



5 At a flow point 270, an outgoing packet has been transmitted at one of the 

6 output ports 112. 



7 



8 The router 110 operates with regard to each packet 140 using a parallel 

9 pipeline. Thus, a first packet 140 is being received while a second packet is being trans- 
1 o mitted while a third packet 140 is having a packet lookup performed. 

11 

12 - -The memory 120. has two regions (a packet buffer region for incoming and 

13 outgoing packets 140, and a packet header region for packet header lookup), each of 

u which is intended to be accessed rapidly and often. However, multiple accesses to the 

15 memory 120 do not occur simultaneously; instead they are multiplexed so that accesses 

16 to these regions, are each serviced often by the memory 120, and prioritized so that ac- 

17 cesses to these regions can each be serviced rapidly by the router 110. 

18 

19 In a preferred embodiment, packet reception is performed with relatively 

20 high priority, packet transmission is performed with medium priority, and packet for- 

21 warding lookup is performed with relatively low priority. 

22 

23 Access requests by the receive request circuit 163 have the highest prior- 

24 ity, so that when requests for such accesses are received by the memory 120, they are 

25 processed before requests for accesses by other circuits. Thus, incoming packets are en- 
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tered into and retrieved from the input packet queue 162 as quickly as possible, so that 
queuing at the input ports 111 of the router 1 10 is minimized. 

Access requests by the transmit request circuit 173 have medium priority 
(after requests by the receive request circuit 163 and before requests by the address re- 
quest circuit 183), so that when requests for such accesses are received by the memory 
120, they are processed after requests for accesses by the receive request circuit 163 and 
before requests by the address request circuit 183. Thus, outgoing packets are entered 
into and retrieved from the output packet queue 172 as quickly as possible after incom- 
ing packets are processed. 

Access. requests by the address request circuit 183 have the lowest priority, 
so that when requests for such accesses are received by the memory 120, they are proc- 
essed after requests for access by other circuits. 

Alternative Embodiments 

Although preferred embodiments are disclosed herein, many variations are 
possible which remain within the . concept, scope, and spirit of the invention, and these 
variations would become clear to those skilled in the art after perusal of this application. 
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Claims 

1. A router, including 

at least one port disposed for receiving packets; 

at least one port disposed for transmitting packets; and 

processing circuits integrated into a single monolithic integrated circuit on 
at least one side of a silicon die, said processing circuits including 

means for accessing a shared memory, said means including (a) circuits dis- 
posed for copying packets between at least one region of said shared memory and said 
processing circuits, and (b) circuits disposed for performing packet lookup in at least one 
region of said shared memory, said packet lookup being responsive to packet headers of 
- said -packets.- _ . 

2. A router as in claim 1, wherein said at least one port disposed for re- 
ceiving packets includes a plurality of ports disposed for receiving packets. 

3. A router as in claim 1, wherein said at least one port disposed for 
transmitting packets includes a plurality of ports disposed for transmitting packets. 

4. A router as in claim 1, wherein said circuits disposed for copying 
packets include circuits for receiving packet information in a parallel format and con- 
verting that information into a serial format. 
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5. A router as in claim 1, wherein said circuits disposed for copying 
packets include circuits for receiving packet information in a serial format and convert- 
ing that information into a parallel format. 

6. A router as in claim 1, wherein said means for accessing said shared 
memory includes circuits disposed for multiplexing and prioritizing requests for copying 
incoming packets to said shared memory, requests for copying outgoing packets from 
said shared memory, and requests for performing packet lookup. 

7. A router as in claim 6, wherein said circuits disposed for multiplex- 
ing and prioritizing assign relatively high priority to said requests for copying incoming 
packets to-said shared-memory. _ 

8. A router as in claim 6, wherein said circuits disposed for multiplex- 
ing and prioritizing assign relatively low priority to said requests for copying outgoing 
packets from said shared memory. 

9. A router as in claim 6, wherein said circuits disposed for multiplex- 
ing and prioritizing assign priority to said requests in a manner so as to maximize 
throughput and minimize routing latency. 

10. Apparatus including 

a shared memory, said shared memory including packet buffers for packets 
and packet lookup information; 
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a router coupled to said shared memory, said router including processing 
circuits integrated into a single monolithic integrated circuit on at least one side of a sili- 
con die, said processing circuits including 

means for accessing said shared memory, said means including (a) circuits 
disposed for copying packets between at least, one region of said shared memory and 
said processing circuits, and (b) circuits disposed for accessing said shared memory for 
performing packet lookup. 

11. Apparatus as in claim 10, including a processor coupled to said 

router. 

12 ~ - ■ - - - - 12. -Apparatus as in claim 10, wherein said means for accessing said 
shared memory includes circuits disposed for multiplexing and prioritizing requests for 
copying incoming packets to said shared memory, requests for copying outgoing pack- 
ets from said shared memory, and requests for performing packet lookup. 

13. Apparatus as in claim 12, wherein said circuits disposed for multi- 
plexing and prioritizing assign relatively high priority to said requests for copying in- 
coming packets to said shared memory. 

14. Apparatus as in claim 12, wherein said circuits disposed for multi- 
plexing and prioritizing assign relatively low priority to said requests for copying out- 
going packets from said shared memory. 
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1 15. Apparatus as in claim 12, wherein said circuits disposed for multi- 

2 plexing and prioritizing assign priority to said requests in a manner so as to maximize 

3 throughput and minimize routing latency. 

4 

5 16. Apparatus as in claim 12, wherein said circuits disposed for copying 

6 packets include circuits for receiving packet information in a parallel format and con- 

7 verting that information into a serial format. 

8 

9 17. Apparatus as in claim 12, wherein said circuits disposed for copying 

I o packets include circuits for receiving packet information in a serial format and convert- 

I I ing that information into a parallel format. 

1 2 V -------- 

13 18. A method, including the steps of 

14 receiving packets using at least one input port; 

1 5 transmitting packets using at least one output port; and 

16 routing said packets from said at least one input port to said at least one 

17 output port, using processing circuits integrated into a single monolithic integrated cir- 

1 8 cuit on at least one side of a silicon die. 

19 

20 19. A method, including the steps of 

21 receiving packets using at least one input port; 

22 transmitting packets using at least one output port; 

23 copying packets between at least one region of a shared memory and a set ' 

24 of processing circuits integrated into a single monolithic integrated circuit on at least one 

25 side of a silicon die; and 
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performing packet lookup using said shared memory and said processing 

circuits. 

20. A method as in claim 19, wherein said at least one input port in- 
cludes a plurality of ports disposed for receiving packets. 

21. A method as in claim 19, wherein said at least one output port in- 
cludes a plurality of ports disposed for transmitting packets. 

22. A method as in claim 19, wherein said step of copying packets in- 
cludes the steps of receiving packet information in a parallel format and converting that 
information into a serial format. 

23 . A method as in claim 19, wherein said step of copying packets in- 
cludes the steps of receiving packet information in a serial format and converting that 
information into a parallel format. 

24. A method as in claim 19, wherein said steps of copying packets and 
performing packet lookup include the steps of multiplexing and prioritizing requests for 
copying incoming packets to said shared memory, requests for copying outgoing pack- 
ets from said shared memory, and requests for performing packet lookup. 

25. A method as in claim 24, wherein said steps of multiplexing and pri- 
oritizing assign relatively high priority to said requests for copying incoming packets to 
said shared memory. 
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26. A method as in claim 24, wherein said steps of multiplexing and pri- 
oritizing assign relatively low priority to said requests for copying outgoing packets 
from said shared memory. 

27. A method as in claim 24, wherein said steps of multiplexing and pri- 
oritizing assign priority to said requests in a manner so as to maximize throughput and 
minimize routing latency. 

28. A method, including the steps of 

recording packets and packet lookup information in a shared memory; 

- - coupling processing^circuits to said shared memory, said processing circuits 

being integrated into a single monolithic integrated circuit on at least one side of a sili- 
con die, said processing circuits including (a) circuits disposed for copying packets be- 
tween at least one region of said shared memory and said processing circuits, and (b) cir- 
cuits disposed for accessing said shared memory for performing packet lookup. 

29. A method, including the steps of 

recording packets and packet lookup information in a shared memory; 

copying packets between at least one region of said shared memory, and 
accessing said shared memory to perform packet lookup, using processing circuits which 
are integrated into a single monolithic integrated circuit on at least one side of a silicon 
die. 
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30. A method as in claim 29, including coupling a processor to said 
processing circuits. 

31. A method as in claim 29, wherein said steps of copying packets and 
accessing said shared memory include the steps of multiplexing and prioritizing requests 
for copying incoming packets to said shared memory, requests for copying outgoing 
packets from said shared memory, and requests for performing packet lookup. 

32. A. method as in claim 31, wherein said steps of multiplexing and pri- 
oritizing assign relatively high priority to said requests for copying incoming packets to 
said shared memory. 

33. A method as in claim 31, wherein said steps of multiplexing and pri- 
oritizing assign relatively low priority -to said requests for copying outgoing packets 
from said shared memory. 

34. A method as in claim 31, wherein said steps of multiplexing and pri- 
oritizing assign priority to said requests in a manner so as to maximize throughput and 
minimize routing latency. 

35. A method as in claim 31, wherein said steps of copying- packets in- 
clude the steps of receiving packet information in a parallel format and converting that 
information into a serial format. 
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36. A method as in claim 31, wherein said steps of copying packets in 
elude the steps of receiving packet information in a serial format and converting that in 
formation into a parallel format. 



WO 99/17182 



1/2 



PCT/US98/20627 



^4o 



Ul 



J T 



t4- 



T- 1 



loo 



WO 99/17182 



PCTAJS98/20627 



2/2 



V 




Ql>{ 



1ZA 



"L 



^2. 




" 1 




This Page is Inserted by IFW Indexing and Scanning 
Operations and is not part of the Official Record 



Defective images within this document are accurate representations of the original 
documents submitted by the applicant. 

Defects in the images include but are not limited to the items checked: 

□ BLACK BORDERS 

□ IMAGE CUT OFF AT TOP, BOTTOM OR SIDES 



□ BLURRED OR ILLEGIBLE TEXT OR DRAWING 

□ SKEWED/SLANTED IMAGES 

□ COLOR OR BLACK AND WHITE PHOTOGRAPHS 

□ GRAY SCALE DOCUMENTS 

□ LINES OR MARKS ON ORIGINAL DOCUMENT 

□ REFERENCE(S) OR EXHIBIT(S) SUBMITTED ARE POOR QUALITY 

□ OTHER: 

IMAGES ARE BEST AVAILABLE COPY. 
As rescanning these documents will not correct the image 
problems checked, please do not report these problems to 
the IFW Image Problem Mailbox. 



BEST AVAILABLE IMAGES 




FADED TEXT OR DRAWING 



